The present invention relates to the field of electronic testing, and more particularly to a method and system for designing, administering and taking tests through a computer network or other information distribution media.
Tests of human knowledge, understanding and preferences are well known. In educational environments, they are typically used as indicators for measuring human performance and our knowledge. Test results-may be used, for example, to screen job applicants, assign grades to students and to gain insight about human abilities, strengths and weaknesses.
A number of computerized query systems are known, including certain educational testing systems, on-line surveys, client-server systems, search engines, Internet, intranet and World Wide Web-based systems, educational toys, and the like.
Typically, these systems allow a test administrator to define the queries, testing parameters and response parameters. Some of these systems may also allow randomization of queries and adaptive testing.
Often, the test or query may be considered a logical unit, with a plurality of requested information components. The structure is typically defined at the top level, i.e., the test definition logical structure is defined unitarily. This leads to difficulties in altering the structure of the test, as it is difficult to define different local parameters for some of the questions in such a structure. Further, this makes test component reuse difficult, as the testing parameters need be included with the sub-component, a function not typically supported in known systems. Therefore, while test content may be portable, test structure must usually be redefined for each version or iteration. This structure also makes optimization of overall test structure difficult, as global parameters are typically a compromise from desired parameters for individual portions of a test.
Prior electronic testing and instructional systems are disclosed in U.S. Pat. Nos. 5,657,256; 5,618,182; 5,597,312; 5,577,919; 5,565,316; 5,545,044; 5,513,994; 5,496,175; 5,441,415; 5,437,555; 5,437,553; 5,433,615; 5,372,507; 5,334,326; 5,261,823; 5,259,766; 5,211,564; 5,211,563; 5,204,813; 5,195,033; 5,180,309; 5,176,520; 5,170,362; 5,059,127; 5,011,413; 5,002,491; 4,978,305; 4,895,518; 4,798,543; 4,877,408; 4,793,813; 4,764,120; 4,671,772; 4,486,180; and 3,654,708, each of which is expressly incorporated herein by reference in its entirety.
The system and method according to the present invention provides an automated testing system allowing design and administration of hierarchical testing scheme. This scheme eases the burden of creating and giving out tests, and allows a fine grain of control over testing parameters. This degree of control is optional, and therefore the construction of a test is not made unnecessarily complex. In fact, in a preferred embodiment, parameters from one portion of a test or from a separate test are readily copied or linked to another portion or test. Therefore, the test designer has the ability to focus on the quality and content of the tests and reliability of the test results, while avoiding redundant tasks.
The present invention overcomes the difficulties of the prior art by providing logical units (i.e., xe2x80x9cbuilding blocksxe2x80x9d) within an overall presented information stimulus, volitional response system. These building blocks can be grouped together in various sequences to create a question and answer response system. Each building block is independent of the others and contains parameters that determine its functionality and implementation. Each building block also contains information about how it relates to the other building blocks. Some systems that can be constructed from these building blocks include educational tests, quizzes, surveys, questionnaires and trivia quizzes. However, any other system that can be formulated as a sequence of questions and answers can also be built from these building blocks (i.e., scientific experiments). Each logical unit or xe2x80x9csectionxe2x80x9d (also known as xe2x80x98building blockxe2x80x99) has a set of parameters which define, for example, an interrelation of that section with other sections; grading instructions; adaptive aspects; allowable timing, sequencing and repetition; security; and randomization. Thus, the overall test structure is not required to define these aspects for each section of the test. This allows a fine level of granularity in defining the overall testing function, while allowing optimization of individual sections, easing reuse of sections in other tests while maintaining the testing parameters, and assisting in the incremental formulation of a test structure.
A section consists of one or more elements, being subject to the common parameter set. The element, in turn, may include any number of questions, i.e., zero or more. One or more sections may be grouped into a logical display set, termed a xe2x80x9cpartxe2x80x9d.
The test part, for example, is transmitted as a single logical unit from a server, which assembles each section, as necessary, based on the parameters, and then groups the sections, based on other parameters, as a part. The server then awaits receipt of responses, which are in a form defined by the section parameters, and interprets the responses in accordance with the section parameters, for output. The output may be subject to further analysis, such as individual and group statistics, feedback to the test taker, or the like. In this case, the server is not burdened with continual monitoring of each user, and thus may serve a large number of test takers simultaneously, without overload. The server is a transactional based device, servicing each test taker as needed to register, transmit a test part, and receive results.
Because each test section includes its own local parameters, a wide variety of data types may be included in a section, without requiring the test structure as a whole to be unnecessarily complex. For example, Hypertext markup Language (HTML) 3.0 is presently preferred to define the display of each section, which is collected for each section into a Web page as the part. As such, full data type integration, hyper-linking, and display formatting are supported. For example, this allows a section to include a pointer to a file containing a video clip, which is downloaded from a different Web server. This use of an open hyper-linking protocol, such as HTML 3.0, to define the presentation in a testing environment is new, and provides significant advantages, including the ability to reference data which may change continuously or at variable times.
By employing a standard language such as HTML, the system is operable over intranets or the Internet using standard Web browsers, such as Microsoft Internet Explorer (MSIE), e.g., MSIE 3.02, Netscape Navigator, e.g., Navigator 4.01, or the like, on standard client computing platforms. However, WebTV(copyright) or an Internet appliance, such as Internet capable personal communication service (PCS) devices may also administer tests. Thus, the benefits of the present invention are possible using standard server and client systems, with a custom Web server (or other xe2x80x9cTestxe2x80x9d server) to generate test parts and receive responses, and the system does not necessarily require customized hardware. Of course, the system need not be open, and proprietary hardware, protocols and software may be employed. The system is also extensible and not necessarily linked to HTML 3.0, and thus may employ other or future communication or presentation standards, as desired or as they become available.
Since a preferred embodiment of the present invention operates using standard Web browsers over the Internet, a section or part of a test may comprise:
A browser plug-in
TCP/IP socket applications running on the client
JAVA applets or ActiveX objects
Any other systems that enhance the client functionality beyond existing HTML functionality
The presented content may include audio, video, multimedia, or other information. Because the test taker typically employs a general-purpose computer as the platform, other facilities of the platform may be employed to assist the test taker. Thus, the resources on the computer (such as a calculator) and the Internet (such as other web pages) may be used to aid the test taker in taking the test.
It is noted that it may also be desired to seek to prevent a user from viewing or receiving information other than that provided by the test designer. This may be effected, for example, by providing a closed software application that prevents the user from employing the full capabilities of the client computer. A video camera, for example ITU H.323 videoconferencing or still image format such as JPEG or GIF, and screen content monitor application may also be provided to proctor the test. The student keyboard may also be monitored for suspicious activity or hiatuses in activity.
It is noted that, while a preferred embodiment of the present invention allows multiple remote users (clients) to interact with a central query server (host), the system may, in fact, be constituted for a single user on a non-networked computer system. In this case, the client and host systems are separated by a distinction in tasks, the client system being primarily directed to human user interface tasks and the host being directed to database and application processing tasks.
Therefore, the system provides a database of the elemental sections (xe2x80x98building blocksxe2x80x99) of the test, each including a content portion and a set of parameters. The sections are assembled into a logical unit by an application program, which may be a part of the database retrieval system, a distinct host or client system which receives the section information from a host. The logical unit is transmitted to a client for presentation and interaction with the user. The client may include enhanced functionality defined by the logical unit, but outside the scope of the transmission. The client obtains information relating to the sections from the user or through interaction with the user, and transmits results or responses back to the server, for analysis. The server may then transmit another logical unit to the client, which may have no defined relation to the prior logical unit, or have a logical relation defined by the parameters or global test parameters. It is noted that the host or server and client may be merged into a single physical apparatus, or be distinct and communicate through a communication medium.
In a preferred embodiment, the client processes the user responses and delivers the entirety of the responses to the server at one time; however, this is not a limitation of the system as a whole. In order to free the server for various activities, without storage of interim results, each part of the test may thus include prior results. Therefore, the test need only be submitted once for grading. In order to prevent tampering with answers where the answer is already submitted (but not entered), a code may be transmitted with the test part that indicates a status of the prior results. Any change in these prior results by, for example, manual change or error in transmission will result in an indication of tampering. In order to reduce the possibility of data transmission errors, an error correction and detection code (EDC) may be provided.
The query need not be of an academic or demographic nature, and therefore the more general aspect of the system seeks to determine a human response to a stimulus. This stimulus is described by a record in a database, which also includes parameters, which may be intrinsic, defining the content, extrinsic, defining a relation of the record to other records, or hybrid, defining the content based on references to external elements. The content portion, in turn, may also have internal or external references, although these references are not necessarily processed at the server level, and may be passed to the client for direct communication with the source (server) of the external reference. The user, interacting with the client, experiences the stimulus, and a response recorded. As noted, the stimulus need not be explicitly defined in the content of the server-based database, but rather may be referenced by that database. Typically, the response is manually recorded, but automated data acquisition systems may also be used. While the system is open to broad classes of stimuli and responses, in a preferred embodiment, the stimulus is a text or multimedia presentation supported by a desktop computer with communication link, and the response is gathered through a graphic user interface from a human user.
The responses may be, for example, in the form of multiple choice answers, buttons, check boxes or dialog boxes, or may be sliders, image maps, or other types of xe2x80x9ccontinuousxe2x80x9d response inputs. Typically, the user interacts with a graphic user interface, and thus a pointing device input and keyboard may be employed to define the response. The stimulus is typically graphic or semantic information displayed on a screen.
The present invention may be differentiated from known systems in that sections of a query (xe2x80x98building blocksxe2x80x99) may have individual sets of parameters, allowing a fine granulation of control. The present invention may also be differentiated from known systems in that the query is delivered to the respondent in a form that may include internal and/or external references to content.
In a preferred embodiment, the present system allows for the interactive and secure development and production of tests over the Internet using a standard web browser. This program can be used along with any standard ODBC compliant database and any standard web browser that supports the HTTP protocol. The server stores various statistics and other information about the test, the takers of the test and the questions on the test.
The system gives the test developer the freedom to concentrate on creating the test and the content of the test rather than worrying about the way the test is going to be given and how it is going to be produced. The way the test is going to be given and produced are optimized and standardized by the system and method according to the present invention, which automates the test presentation. The system sets an environment where interesting and unique kinds of tests can be given in a dynamic medium. This includes the ability to give multimedia-based questions that consist of video clips, sound and pictures. It also includes the ability to use the resources on the computer (such as a calculator) and the Internet (such as other web pages) to aid the test taker in taking the test.
The xe2x80x9cInteractive Testing Systemxe2x80x9d (xe2x80x9cITSxe2x80x9d) uses the following, conceptual hierarchical model:
Test Administrator(s)xe2x86x92Test Designer(s)xe2x86x92Test(s)xe2x86x92Section(s)xe2x86x92Question(s)xe2x86x92Choice(s)
Test Administratorsxe2x86x92Test Taker(s)
Test Designer(s)xe2x86x92Test Taker(s)
Test Part: Sectionxe2x86x92[Optional Break]xe2x86x92Sectionxe2x86x92[Optional Break] . . .
The ITS system provides the following features:
Allows for multiple test designers to design any number of their own password-protected tests
Allows each test designer to develop any number of tests
Allows each test to have zero or more sections on it (i.e., zero or more test parts)
Allows each section to have zero or more questions in it
Allows for a test to be divided into parts and restricts the number of xe2x80x9cvisitsxe2x80x9d to those parts
Allows for the computation of how much cumulative time has been spent on different parts of a test
Allows multiple sections to be given on the same test part
Allows for breaks to be given during the test
Allows for the analysis of questions and answers to see how many people chose one response over another
Gives tests securely, automatically and accurately:
Checks for test tampering
Disallows xe2x80x9creloadingxe2x80x9d of prior test parts by using the xe2x80x9creloadxe2x80x9d button on a standard web browser
Automatically gives and grades test. Automatic grading saves hours of potential work.
Records various statistics about a test
Allows for the questions within a section to be:
Asked in random order
Asked in sequential order
Asked a certain number of times (i.e., ask ten questions on this section)
Multiple choice, one answer
Multiple choice, one correct answer questions
Multiple choice, one or more correct answer questions
Multiple choice, zero or more correct answers questions
Fill-In-The-Blank questions
Allows test takers to:
View test results (number right, wrong or no answer; scores: average, low, high)
Compare how they did to the test median and standard deviation as well as the test mean, low score and high score
See what answers they put to the questions and view explanations to wrong answers
Allows for the development of xe2x80x9cRegularxe2x80x9d tests and xe2x80x9cProgressivexe2x80x9d (also known as xe2x80x9cAdaptivexe2x80x9d) tests where xe2x80x9cProgressivexe2x80x9d tests get harder or easier depending on how the user did on the last test
Allows for the development of xe2x80x9cRegularxe2x80x9d sections and xe2x80x9cProgressivexe2x80x9d sections within a test
Penalizes for xe2x80x9cguessedxe2x80x9d answers to questions
Allows for scores to be given within a range of scores. Scores can be in xe2x80x9cinvertedxe2x80x9d order where higher scores are given to more incorrect answers (xe2x80x9cnegative rewardxe2x80x9d. . . you must get all the answers wrong to get the best possible test score)
Allows sections of a test to be copied to other tests
Allows questions in a test section to be copied to other sections on different tests
Records test designer information:
Name
Logon Identifier
Password
Address, City, State, Zip code and Phone Number
Fax Number
Email Address
Records test information:
Unique Test Identifier
Whether to give sections on a test in random order
Minimum number of sections to give on test
Maximum number of sections to give on test
The lowest score to be given on a test
The highest score to be given on a test
The passing score for a test
A URL to go to if the test is passed
A URL to go to if the test is failed
Whether to show answers and explanations to the questions upon test completion
Records section information:
The section order # within test (can be the same as another section in which case one of the sections will be selected at random)
The section heading (also known as the section identifier, label or name)
The section introduction
The section ending
The section weight relative to 1.0
The number of questions within the section
The minimum number of questions to ask (within difficulty level if any) on the section
The maximum number of questions to ask (within difficulty level if any) on the section
Whether to ask questions in random or sequential order
What order to ask questions within the section in
At one particular difficulty level
By increasing difficulty level
By decreasing difficulty level
By random difficulty level
By Ignoring the difficulty level
What order to ask questions within a difficulty level (if not ignoring difficulty level):
Random
Sequential (order entered)
The maximum amount of time that a person can work on a section
The minimum amount of time that a person must work on a section
Whether to given the next section along with the current section or in a different part on the test
The maximum amount of time to pause (if any) after a section has been given
The minimum amount of time to pause (if any) after a section has been given
The passing section score, low section score and high section score (for xe2x80x98Progressivexe2x80x99 tests)
The next section order number to go to if get greater than, less than or equal to the passing section score
Records question information:
The question to ask (can include HIML code)
The explanation for the question (can include HTML code)
The question order number within the section
Question difficulty level
The number of points for getting the question right, wrong or not putting an answer to the question
Whether to give the choices for a question in random order or sequential order
The actual choices to the question to be displayed (for multiple choice)
Records test taker information:
Name
Logon Identifier
Password
Address Information
Phone number
Fax number
Email address
Maximum number of tests allowed to take total
Skeleton file used when taker logs on to take the test
Type of test taker:
Regular
Template for other users
Permissions:
What test taker is allowed to do when they log on to see test results
View fields above
Modify fields above
What happens when a test taker logs on to take a test:
Fields above are required to be entered by taker
Fields above are verified with matching entries in database
Fields above will replace current information in the database
Who is allowed to take which tests how many times and between which dates
Logging:
What level of results to record for a test taker:
Test
Section
Questions
Records test statistics:
Number of times test has been given
Number of times test has been graded
Number of times test has been passed/failed
Scores:
Average, Total, Maximum, Minimum, Mean, Median, Standard Deviation
Test Duration:
Average, Total, Maximum, Minimum, Mean, Median, Standard Deviation
Number Of Times Test Has Been Taken:
Average, Total, Maximum, Minimum, Mean, Median, Standard Deviation
Records test section statistics:
Number of times section has been asked
Number of times section has been graded
Section Duration:
Average, Total, Maximum, Minimum, Mean, Median, Standard Deviation
Section Visits:
Average, Total, Maximum, Minimum; Mean, Median, Standard Deviation
Records question statistics:
Number of times question has been asked or graded
Number of times question has been answered correctly, incorrectly or not answered
Records test taker statistics and results:
xe2x80x9cPaybackxe2x80x9d any test that a user has taken
a Records grades, answers to questions, number of times they have taken the test
Allows taker to see how they did compared to other users
Records the amount of time a user has spent on each part of the test
Records the date and time a user takes the test out on
Allows test designers to:
Maintain a list of private test takers:
Create
Modify
Delete
Email
Cancel tests outstanding (i.e., previously given out)
Assign test takers (created by designer or administrator) to take tests created
Give out tests to individual users that they create
Allows test database administrator to:
Become ANY test designer and perform ALL functions associated with that test designer
Maintain designer accounts:
Create
Modify
Delete
Email designer
View test database statistics:
Total number of tests given, graded, passed
Total number of questions given, graded, passed
Cancel ANY outstanding tests being given out by any designer
Create xe2x80x9cGlobalxe2x80x9d users who are able to take ANY test developed by ANY designer
Allows test designer to control exactly what test taker views when taking a test
The ITS allows a person to interactively create and give out tests securely on the Internet using a standard web browser interface. The use of the ITS can be broken into the following areas of functionality:
1 xe2x80xa2 Test Design and Development
2 xe2x80xa2 Test Taking (xe2x80x9cUsing The Testxe2x80x9d)
3 xe2x80xa2 Statistics, Reports and Analysis of Test Results
The steps necessary to design and develop a test can be broken down as follows:
1 xe2x80xa2 Test Planning and Preparation
2 xe2x80xa2 Test Designer Information
3 xe2x80xa2 Test Information
4 xe2x80xa2 Sections For Test Information
5 xe2x80xa2 Questions On A Section
The ITS can be used as a tool to provide instruction and give out tests on the Internet. It does so by implementing a xe2x80x9cTest-Centricxe2x80x9d academic environment. This is a form of learning where the test is at the heart of providing the content. The test becomes the focal point for the lesson and the questions on the test point to areas on the web where the answer to the questions can be learned about.
This xe2x80x9cTest-Centeredxe2x80x9d approach gives the test taker more freedom in deciding how they want to learn. If they already know the answer to a question, they can skip the instructional portion and go on with the other questions. If they don""t know the answer, they can link to information on the topic in another area of the web.
The ITS system therefore allows a test designer to:
Produce tests with multiple-parts, one part or a random number of parts
Create tests with time limits (including timed pauses between parts)
Give out the sections of a test in any order (even in random order)
Make tests with a fixed number of sections or a random number of sections
Produce tests with different questions for different takers
Limit who can take your test by password protecting your tests by taker
Re-use parts of tests in other tests
Decide whether to show answers after taking a test
Send test xe2x80x98webxe2x80x99 instructions along with your test
This, in turn, allows the designer to gain complete control of your questions, answers and grading, to:
Create multiple choice questions (one answer correct or zero-or-more answers correct)
Design fill-in-the-blank questions
Assign point values to questions for correct answers, wrong answers, or no answers
Ask questions in any order . . . fixed or random
Choose questions randomly from a pool of questions
Ask a fixed set of questions
Re-use questions and answers from previously designed tests
Give choices to questions in a random order
Deduct points for questions that were guessed
Scale test scores to be within any range
Allow test takers to view test statistics immediately (see the xe2x80x9cbell shaped curvexe2x80x9d)
Give web links for explanations of answers or missed questions
The ITS system also maintains vital test statistics, such as:
Know who took your test, when they took it, what they got right (or wrong) and how many times they took it
Compute the total number of times your test was given
See what questions were missed or correctly answered the most or least
Analyze the effectiveness of tests by computing grades without grading
Find out how many students in your class are having problems with certain questions
Monitor progress of individual test takers over the course of years of testing.
The ITS system has various reporting formats, including:
Automatically generate notes to test takers (students, job applicants, . . . )
Generate notes and periodical report cards home to parents
Give feedback to upper management about how people have done in the past
Because, in a preferred embodiment, the ITS system resides on the Internet, the system allows the test designer and test taker to take advantage of the Internet resources to create new tests, for example, to:
Show answers to questions with explanations on the web
Give questions based on the content of another web site
Show video clips or play sound files for xe2x80x9cmulti-mediaxe2x80x9d questions
Give instructions using web pages
Other advantages of the automated ITS system include:
No need to grade exams manually
Allows more time for teaching and less time copying tests for students
Can utilize Internet resources for students to do research
Ability to monitor your test takers to gain immediate feedback and assist if need be
Permanent record of past performance kept. See how students and test taker have progressed over the years.
The ITS system also provides a number of security features, including:
Allowing the test designer to decide who can take your test(s), the number of times they can take them and when they can take them.
Password-protection of tests to disallow them from being viewed by others when they are designed.
Tampering with tests is not allowed (answers may be checked to ensure that they aren""t changed).
It is therefore an object of the invention to provide a human response testing system, comprising a host, storing a plurality of testing parameters, the testing parameters being divided into a plurality of sections; and a terminal, communicating with the host through a communication system, adapted for interaction with the human to acquire responses, wherein the host transmits information defining at least one of the sections to the terminal, the terminal presents inquiries to the human defined by the section for receipt of responses, and the terminal transmits information relating to the responses to the host; and wherein the host processes the information relating to the responses to inquiries defined by the section and outputs the processed information from the host.
It is another object according to the invention to provide a human response testing system, comprising a host, storing a plurality of testing parameters, the testing parameters being divided into a plurality of sections; and a terminal, communicating with the host through a telecommunications link, adapted for interaction with the human to acquire responses, wherein the host transmits information defining one of the sections to the terminal, including designated queries and context information, and defines information representative of a state of the section before transmission, the terminal presents inquiries to the human defined by the section for receipt of responses to the queries, and the terminal transmits an answer including information relating to the responses and a modified state to the host, the host processing the responses to determine an information content and determining a change between the defined state and the modified state to indicate changes between the transmitted context information and the information relating to the responses.
It is a still further object according to the invention to provide an interactive query server system, comprising a database storing a plurality of query sections, each section including parameters defining an graphic layout, the graphic layout defining display elements from a plurality of separately stored data records, the query server being adapted to generate, based on the parameters, query sequence sets, each set having at least one query section; transmit a query sequence set as a logical unit to a client system; receive responses of a user to the query sequence set from the client system; and process the received responses to produce an output.
It is also an object according to the present invention to provide an interactive query server system, comprising a database storing a plurality of query sections, each section including parameters defining a relation with other query sections, the query server being adapted to generate, based on the parameters, query sequence sets, each set having at least one query section; transmit a query sequence set as a logical unit to a client system; receive responses of a user to the query sequence set from the client system; and process the received responses to produce an output.
According to an object of the invention, the host may transmit information defining at least one of the sections to the client system, the client system presents inquiries to the user defined by the section for receipt of responses, and the client system transmits information relating to the responses to the query server; and wherein the query server processes the information relating to the responses to inquiries defined by the section and outputs the processed information from the query server. The query server may transmit information defining one of the sections to the client system, including designated queries and context information, and defines information representative of a state of the section before transmission, the terminal presents inquiries to the user defined by the section for receipt of responses to the queries, and the client system transmits an answer including information relating to the responses and a modified state to the query server, the query server processing the responses to determine an information content and determining a change between the defined state and the modified state to indicate changes between the transmitted context information and the information relating to the responses. The query server and the client system may be separate systems communicating over a telecommunications network (e.g., local area network, wide area network, intranet, Internet, wireless data communication network), or located on the same physical hardware. The communication may be open or encrypted, and the client software may be open (compatible with a large number of third party hardware and software products) or closed (limited in operation to a particular environment or type of environment)
According to another object of the present invention, the query server formats a test as an HTML document, which may be, for example, read and responded to through a so-called web browser. The query sequence may include plain text, references to objects on the query server, references to objects on the client, or references to objects on external servers. These objects may be, for example, text, graphics, video (e.g., compressed video, such as MPEG, QuickTime or AVI), audio, multimedia, or control signals for hardware devices. While a standard computing platform is a preferred client system, the display device may also be a video receiver
It is a further object of the invention according to a preferred embodiment to service a plurality of clients with a single query server.
It is another object according to the present invention to provide a system in which the query sequence set comprises query content and parameters for processing of the query content. The query server, after receiving responses of the user to the query sequence set from the client system, may transmit a second query sequence set. The processing parameters for the first query sequence set may differ from the processing parameters for the second query sequence set. According to one embodiment, the user responses to test parts are cumulative, so that the query server need only record the test responses once. An encoded message may be provided in each test part to indicate unwarranted changes made to a previously submitted part, thereby allowing answers to be effectively locked. The encoded message may be a parity check, cyclic redundancy check, or other known encoding scheme.
The query server may perform a statistical analysis of the received response(s), and may also perform chronological measurements of the received responses. The query server may also produce the query sequence sets xe2x80x9con demandxe2x80x9d, i.e., formulate them at the time of transmission. This allows adaptive and random order tests to be administered.
The administration of the test takers and the tests may be performed either locally at the query server device, or remotely through a network system.
These and other objects of this invention will become more apparent from the following description and accompanying drawings in which like reference numerals depict like elements. Other details of the system and the method of the present invention are set out in the following description.